Approximate Joins for Relational Data

Krommydas, Ioannis, Evagelos, Georgia. MSc, Computer Science Department, University of Ioannina, Greece. June, 2008. Approximate Joins for Relational Data. Thesis Supervisor: Vassiliadis Panos. Relational databases often contain duplicate data entries. This may occur due to a variety of reasons, such as typographical errors, multiple conventions for recording database fields or other noise sources. Duplicate detection is a crucial procedure, especially for large databases. In this thesis, we present a method that extends the state-of-the-art method for duplicate detection. Given a database holding valid data information, we classify each input tuple as a new tuple, or as an existing tuple. The proposed method uses an effective algorithm for determining a set of candidate reference tuples. For each candidate reference tuple, we use appropriate similarity metrics in order to decide whether the input tuple matches a reference tuple. The whole procedure is accelerated via trie data structures for caching the frequent input tuples. Finally, we present a number of experiments evaluating the effectiveness of our method and state a comparative study with the state-of-the-art method.